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Abstract 

We give an explicit (in particular, deterministic polynomial time) construction of 
subspaces X C M. N of dimension (1 — o(l))N such that for every x E X, 



(log AO" ^ 1 ^ 10 ^^ 1Mb «S IMIi ^ VN\\x\\ 2 . 

If we are allowed to use ivVlogiogJV ^ N o(l) ranc i om bits and dim(X) > (1 - rj)N for 

any fixed constant 77, the lower bound can be further improved to (log iV) _ °( 1 )\/]V||a;||2- 

Through known connections between such Euclidean sections of l\ and compressed 

sensing matrices, our result also gives explicit compressed sensing matrices for low 

compression factors for which basis pursuit is guaranteed to recover sparse signals. Our 

construction makes use of unbalanced bipartite graphs to impose local linear constraints 

' on vectors in the subspace, and our analysis relies on expansion properties of the graph. 
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1 Introduction 



Classical results in high-dimensional geometry [131 123] state that a random (with respect to 
the Haar measure) subspace X C M. N of dimension eN [13] or even (1 — e)N [23] is an almost 
Euclidean section in in the sense that vJV||aj||i and ||x||2 are within constant factors, 
uniformly for every i6l. Indeed, this is a particular example of the use of the probabilistic 
method, a technique which is now ubiquitous in asymptotic geometric analysis. 

On the other hand, it is usually the case that objects constructed in such a manner 
are very hard to come by explicitly. Motivated in part by ever growing connections with 
combinatorics and theoretical computer science, the problem of explicit constructions of 
such subspaces has gained substantially in popularity over the last several years; see, e.g. 
[3"B1 Sec. 4], [30, Prob. 8], [221 Sec. 2.2]. Indeed, such subspaces (viewed as embeddings) are 
important for problems like high-dimensional nearest-neighbor search [Tj5] and compressed 
sensing [10], and one expects that explicit constructions will lead, in particular, to a better 
understanding of the underlying geometric structure. (See also the end of the introduction 
for a discussion of the relevance to compressed sensing.) 

1.1 Previous results and our contributions 

If one relaxes the requirement that dim(X) = Q(N) or allows a limited amount of randomness 
in the construction, a number of results are known. In order to review these, we define the 
distortion A(X) of X C R N by 

A(X) = VN ■ max f^. 

O^reX \\x\\i 

In the first direction, it is well-known that an explicit construction with distortion 0(1) and 
dim(X) = Q(yN) can be extracted from Rudin [32] (see also [26] for a more accessible ex- 
position). Indyk [20] presented a deterministic polynomial-time construction with distortion 
1 + o(l) and dim(X) ^ exp{0{lo N glogN)2 y 

Another very interesting line of research pursued by various authors and in quite different 
contexts is to achieve, in the terminology of theoretical computer science, a partial deran- 
domization of the original (existential) results. The goal is to come up with a "constructive" 
discrete probabilistic measure on subspaces X of M. N such that a random (with respect to 
this measure) subspace still has low distortion almost surely, whereas the entropy of this 
measure (that is, the number of truly random bits necessary to sample from it) is also as 
low as possible. 

Denoting by Ak,N a random k x N sign matrix (i.e. with i.i.d. Bernoulli ±1 entries), 
one can extract from the paper [23] by Kashin that ker(Ak,N), a subspace of codimension 
at most k has, with high probability, distortion \jN/k ■ polylog(iV/fc). Schechtman [33] 
arrived at similar conclusions for subspaces generated by rows of An-u,n- Artstein-Avidan 
and Milman [2] considered again the model ker^^) and derandomized this further from 
0(N 2 ) to 0(N log N) bits of randomness. We remark that the pseudorandom generator 
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approach of Indyk [19] can be used to efficiently construct such subspaces using 0(iV log 2 N) 
random bits. This was further improved to 0(N) bits by Lovett and Sodin [27]. Subsequent 
to our work, Guruswami, Lee, and Wigderson [IB] used the construction approach from this 
paper to reduce the random bits to 0(N S ) for any 5 > while achieving distortion 2°^^\ 

As far as deterministic constructions with dim(X) = Q(N) are concerned, we are aware 
of only one result; implicit in various papers (see e.g. [TT] ) is a subspace with dim(X) = N/2 
and distortion 0(N 1/4 ). For dim(X) ^ 3JV/4, say, it appears that nothing non-trivial was 
shown prior to our work. 

Our main result is as follows. 

Theorem 1.1. For every rj = rj(N), there is an explicit, deterministic polynomial-time con- 
struction of subspaces X C ~R N with d'\m(X) ^ (l—rf)N and distortion (?7 _1 loglogiV) ( loglogAr ) 

Like in [23], [2j [27], our space X has the form ker^^jv) for a sign matrix A^n-, but in our 
case this matrix is completely explicit (and, in particular, polynomial time computable) . Its 
high-level overview is given in Section 11.2.31 below. 

On the other hand, if we allow ourselves a small number of random bits, then we can 
slightly improve the bound on distortion. 

Theorem 1.2. For every fixed rj > there is a polynomial time algorithm using N 1 ^ loglogN 
random bits that almost surely produces a subspace X C M. N with dim(X) ^ (1 — rj)N and 
distortion (logN) ^. 

1.2 Proof techniques 

1.2.1 Spreading subspaces 

Low distortion of a section X C M. N intuitively means that for every non-zero x G X, 
a "substantial" portion of its mass is spread over "many" coordinates, and we formalize 
this intuition by introducing the concept of a spread subspace (Definition I2.10p . While this 
concept is tightly related to distortion, it is far more convenient to work with. In particular, 
using a simple spectral argument and Kerdock codes [25], [291 Chap. 15], we initialize our 
proof by presenting explicit subspaces with reasonably good spreading properties. These 
codes appeared also in the approach of Indyk [20J , though they were used in a dual capacity 
(i.e., as generator matrices instead of check matrices). In terms of distortion, however, this 
construction can achieve at best 0(N l l 4 ). 

1.2.2 The main construction 

The key contribution of our paper consists in exploiting the natural analogy between low- 
distortion subspaces over the reals and error-correcting codes over a finite alphabet. 

Let G = ({1,2, ... , N}, Vr, E) be a bipartite graph which is <i-regular on the right, and 
let L C R d be any subspace. Using the notation T(j) C {1,2,..., N} for the neighbor set of 
a vertex j e Vr, we analyze the subspace 

X(G, L) = {x e R N : x m E L for every j G Vr}, 
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where for S C [N], x$ € M' 5 ' represents the vector x restricted to the coordinates lying in 
S. In other words, we impose local linear constraints (from L) according to the structure of 
some bipartite graph G. As Theorem 14.21 shows, one can in particular analyze the spreading 
properties of X(G, L) in terms of those of L and the expansion properties of G. 

1.2.3 Putting it together: combinatorial overview 

Our final space X will be of the form X = f] r { ~^ X(Gi, Li) for suitably chosen G{,Lj (see 
the proof of Theorem II. ip . Combinatorially this simply means that we take ki x N sign 
matrices A, such that X(Gi, Li) = ker(Aj) and stack them on the top of one another to get 
our final matrix A^. Moreover, every A4 is a stack of |Vr| copies of the sign matrix A[ 
with ker(A'j) = Li in which every copy is padded with (N — d) zero columns. The exact 
placement of these columns is governed by the graph G that is chosen to satisfy certain 
expansion properties (Theorem 12 .6f) . and it is different in different copies. 

And then we have one more level of recursion: Every Li has the form X(G[, L'A, where 
G'i again have certain (but this time different - see Proposition 12. 71) expansion properties 
and L\ is our initial subspace (see Section H. 2. ip . 

1.2.4 Connections to discrete codes 

Our approach is inspired by Low Density Parity Check Codes (LDPC) introduced by Gallager 
[14] . They are particularly suited to our purposes since, unlike most other explicit construc- 
tions in coding theory, they exploit a combinatorial structure of the parity check matrix and 
rely very little on the arithmetic of the underlying finite field. Sipser and Spielman [35] 
showed that one can achieve basically the same results (that is, simple and elegant construc- 
tions of constant rate, constant relative minimal distance codes) by considering adjacency 
matrices of sufficiently good expanders instead of a random sparse matrix. These codes are 
nowadays called expander codes. Using an idea due to Tanner [37], it was shown in [35] (see 
also [39J) that even better constructions can be achieved by replacing the parity check by a 
small (constant size) inner code. Our results demonstrate that analogous constructions work 
over the reals: If the inner subspace L has reasonably good spreading properties, then the 
spreading properties of X(G,L) are even better. Upper bounds on distortion follow. 

1.3 Organization 

In Section [2], we provide necessary background on bipartite expander graphs and define 
spread subspaces. In Section [3], we initialize our construction with an explicit subspace 
with reasonably good spreading properties. In Section HJ we describe and analyze our main 
expander-based construction. Finally, in Section we discuss why improvements to our 
bounds may have to come from a source other than better expander graphs. 



3 



1.4 Relationship to compressed sensing. 

In [9], DeVore asks whether probabilistically generated compressed sensing matrices can be 
given by deterministic constructions. 

The note [21] makes the connection between distortion and compressed sensing quite 
explicit. If M : R N -> lR n satisfies A(ker(M)) ^ D, then any vector x G M. N with |supp(a;)| < 
j^p can be uniquely recovered from its encoding Mx. Moreover, given the encoding y = 
Mx, the recovery can be performed efficiently by solving the following convex optimization 
problem: min„ eK iv \\v\\i subject to Mv = y. 

In fact, something more general is shown. Define, for x G Mr, the quantity 

^k{ x )x— mm 1 1 a: — w\\\ (1) 

as the error of the best sparse approximation to x. Then given Mx, the above algorithm 
recovers a vector v G M N such that Mx = Mv and \\x — t>|| 2 ^ <7fc ^ >1 , for k = Q(N/D 2 ). 
In other words, the recovery algorithm is stable in the sense that it can also tolerate noise 
in the signal x, and is able to perform approximate recovery even for signals which are only 
approximately sparse. 

Thus our results show the existence of a mapping M : M N — > J3L°( N \ where M is given by 
an explicit matrix, and such that any vector x G M N with |supp(x)| ^ (i og Af)e?togio g io g iv can be 
efficiently recovered from Mx (the stable generalization also holds, along the lines of (CQ)). 
This yields the best-known explicit compressed sensing matrices for this range of parameters 
(e.g. where n »s iV/poly(log N)). Moreover, unlike probabilistic constructions, our matrices 
are quite sparse, making compression (i.e., matrix- vector multiplication) and recovery (via 
Basis Pursuit) more efficient. For instance, when n = N/2, our matrices have only N 2 ~ £ 
non-zero entries for some e > 0. We refer to [2T] for explicit constructions that achieve a 
better tradeoff for n ~ N s , with < 5 < 1. We remark that the construction of [2T] is not 
stable in the sense discussed above (and hence only works for actual sparse signals). 

2 Preliminaries 

2.1 Notation 

For two expressions A, B, we sometimes write A > B if A = Q(B), A < B if A = 0(B), 
and we write A w B if A = 0(B), that is A > B and B > A. For a positive integer M, [M] 
denotes the set {1, 2, ... , M}. The set of nonnegative integers is denoted by N. 

2.2 Unbalanced bipartite expanders 

Our construction is based on unbalanced bipartite graphs with non-trivial vertex expansion. 

Definition 2.1. A bipartite graph G = (Vl, Vr,E) (with no multiple edges) is said to be an 
(N, n, D, <f)-right regular graph if \Vl\ = N, \Vr\ = n, every vertex on the left hand size Vl 
has degree at most D, and every vertex on the right hand side Vr has degree equal to d. 
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For a graph G = (V, E) and a vertex v G V, we denote by Ta(v) the vertex neighborhood 
{w G V | (v,u) G £?} of v. We denote by d„ = |r G (t>)| the degree of a vertex v. The 
neighborhood of a subset 5 C V is defined by r G (S') = [J veS Ta(v). When the graph G is 
clear from the context, we may omit the subscript G and denote the neighborhoods as just 
r» and T(S). 

Definition 2.2 (Expansion profile). The expansion profile of a bipartite graph G = (Vl, Vr, E) 
is the function A G : (0, \Vl\] — > N defined by 

A G (m) = min{|r G (5)| : S CV L , \S\ > m} . 

Note that A G (m) = mm ve y L d v for < m ^ 1. 

For our work, we need unbalanced bipartite graphs with expansion from the larger side to 
the smaller side. Our results are based on two known explicit constructions of such graphs. 
The first one is to take the edge- vertex incidence graph of a non-bipartite spectral expanded 
such as a Ramanujan graph. These were also the graphs used in the work on expander 
codes [351 EH] • The second construction of expanders is based on a suggestion due to Avi 
Wigderson. It uses a result of Barak, et. al. [I] based on sum-product estimates in finite 
fields; see [38j §2.8] for background on such estimates. 

For our purposes, it is also convenient (but not strictly necessary) to have bipartite 
graphs that are regular on the right. We begin by describing a simple method to achieve 
right-regularity with minimal impact on the expansion and degree parameters, and then turn 
to stating the precise statements about the two expander constructions we will make use of 
in Section H] to construct our explicit subspaces. 



2.2.1 Right-regularization 

Lemma 2.3. Given a graph H = (Vl, Vr, E) with \Vl\ = N, \Vr\ = n, that is left-regular 
with each vertex in Vl having degree D, one can construct in O(ND) time an (N, n', 2D, d)- 
right regular graph G with n' 2n and d = \^-~\ such that the expansion profiles satisfy 
A<3(m) ^ A#(m) for all m > 0. 

Proof. Let c? av = ND/n be the average right degree of the graph H and let d = |~d av ] . Split 
each vertex v G Vr of degree d v into [d v /d\ vertices of degree d each, and if d v mod d > 0, 
a "remainder" vertex of degree r v = d v mod d. Distribute the d v edges incident to v to 
these split vertices in an arbitrary way. The number of newly introduced vertices is at most 
X^ev d v /d = nd w jd ^ n, so the number n' of right-side vertices in the new graph satisfies 
n' ^ 2n. 

All vertices except the at most n "remainder" vertices now have degree exactly d. For 
each v G Vr, add d — r v edges to the corresponding remainder vertex (if one exists). Since 
this step adds at most (d — l)n ^ d w n = ND edges, it is possible to distribute these edges 

lr That is, a regular graph with a large gap between the largest and second largest eigenvalue of its 
adjacency matrix. 
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in such a way that no vertex in Vl is incident on more than D of the new edges. Therefore, 
the maximum left-degree of the new graph is at most 2D. 

The claim about expansion is obvious — just ignore the newly added edges, and the 
splitting of vertices can only improve the vertex expansion. □ 

2.2.2 Spectral expanders 

The next theorem converts non-bipartite expanders to unbalanced bipartite expanders via 
the usual edge-vertex incidence construction. 

Theorem 2.4. For every d ^ 5 and N ^ d, there exists an explicit (N' = O(N), n, 2, 0(d))- 
right regular graph G whose expansion profile satisfies Ac(m) ^ min y / 2mN' 



Proof. Let p, q be any two primes which are both congruent to 1 modulo 4. Then there 
exists an explicit (p + l)-regular graph Y = (V, F) with 9( - <? ~^ $C \V\ ^ l? ^ 2 ~ 1 - > and such 
that A2 = ^ 2y/p, where \2(Y) is the second largest eigenvalue (in absolute value) 

of the adjacency matrix of Y [28]. (See [TBI §2] for a discussion of explicit constructions of 
expander graphs.) 

Letting n = \V\, we define a ( ( p+ ^> n ^ n, 2,p + l)-right regular bipartite graph G = 
(Vl,Vr,E) where Vl — E, Vr — V, and (e,v) G E if v is an endpoint of e G F. To 
analyze the expansion properties of G, we use the following lemma of Alon and Chung [JJ. 

Lemma 2.5. IfY is any d-regular graph on n vertices with second eigenvalue A2, then the 
induced subgraph on any set of jn vertices in Y has at most 

2 , M dn 
edges. 

In particular, if S C Vl satisfies \S\ ^ l 2 (p+ and \S\ ^ 2jn-\/p + 1, then |r G (S')| ^ 
771. Stated different, for any S C Vl, we have 

min |2-^/|5|n, \S\\ 



\r G (S)\ > 



2^/p~+l 



Setting N' = we see that A G (m) ^ min ^f^}. 

Now given parameters d ^ 5 and N ^ d, let p be the largest prime satisfying p + 1 ^ d 
and p = 1 (mod 4), and let q be the smallest prime satisfying q ^ q ~ 1 ^ p+1 ^ ^ jV and q = 
1 (mod 4). The theorem follows by noting that for all integers m ^ 3, there exists a prime 
p G [m, 2m] which is congruent to 1 modulo 4 (see [12]). □ 

The expanders of Theorem 12.41 are already right-regular but they have one drawback; 
we cannot fully control the number of left-side vertices N. Fortunately, this can be easily 
circumvented with the same Lemma [2.31 
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Theorem 2.6. For every d ^ 5 and N ^ d, there exists an explicit (N,n,4,Q(d))-right 
regular graph G which satisfies Ac(m) ^ min v " 2I ^ m j . 

Proof. Apply Theorem 12.41 to get a graph with N' ^ N, N' ~ A" vertices on the left, then 
remove an arbitrary subset oi N — N' vertices from the left hand side. This doesn't affect the 
expansion properties, but it destroys right-regularity. Apply Lemma [2.31 to correct this. □ 

2.2.3 Sum-product expanders 

In this section, p will denote a prime, and ¥ p the finite field with p elements. The following 
result is implicit in [4, §4], and is based on a key "sum-product" lemma (Lemma 3.1) from 
[3], which is itself a statistical version of the sum-product theorems of Bourgain, Katz, and 
Tao [5], and Bourgain and Konyagin [6] for finite fields. 

Proposition 2.7. There exists an absolute constant £o > such that for all primes p the 
following holds. Consider the bipartite graph G p = (F 3 , [4] x W p , E) where a left vertex 
(a, b, c) G F 3 is adjacent to (I, a), (2,6), (3,c) ; and (4, a ■ b + c) on the right. Then K Gp {m) ^ 
min |p a9 , m 1 / 3 ^ } . 

Note that trivially |r Gp (,S)| ^ I'S'I 1 ^ 3 , and the above states that not-too-large sets S 
expand by a sizeable amount more than the trivial bound. Using the above construction, we 
can now prove the following. 

Theorem 2.8. For all integers N ^ 1, there is an explicit construction of an (N, n, 8, 0(A^ 2 / 3 ))- 
right regular graph G which satisfies 

A G (m) ^ min{|n a9 ,m 1/3+fo } . 

(Here £o is the absolute constant from Proposition \2.1\) 



Proof. Let p be the smallest prime such that p 3 ^ N; note that N 1 ^ 3 ^ p ^ 2A^/ 3 . Construct 
the graph G p , and a subgraph H of G p by deleting an arbitrary p 3 — N vertices on the left. 
Thus H has A" vertices on left, 4p vertices on the right, is left-regular with degree 4 and 
satisfies, by Proposition 12.71 A#(m) ^ min {p 9 , m 1//3+ ^° }. Applying the transformation of 
Lemma I2T31 to H, we get an (N, n, 8, <i)-right regular graph with d = \^~\ ~ N 2 ^ 3 and with 
the same expansion property. □ 

2.3 Distortion and spreading 

For a vector x G M. N and a subset S C [N] of coordinates, we denote by xs G M' s ' the pro- 
jection of x onto the coordinates in S. We abbreviate the complementary set of coordinates 
[N] \StoS. 

Definition 2.9 (Distortion of a subspace). For a subspace X C M. N , we define 



A(X) = sup -f ll || X|12 . 

x£X \\X i 



7 



As we already noted in the introduction, instead of distortion it turns out to be more 
convenient to work with the following notion. 

Definition 2.10. A subspace X C M. N is (t,e)-spread if for every x G X and every S C [N] 
with \S\ ^t, we have 

\\xsh ^ £ ■ IMh- 
Let us begin with relating these two notions. 
Lemma 2.11. Suppose X C Mr. 
a) If X is (t,e) -spread then 

A(X)^M-e- 2 ; 



b) conversely, X is ( 2 Apf) 2 ' 4A(x) ) - s P rea d- 



Proof, a) . Fix x G X; we need to prove that 

\\x\\i ^ Vte 2 \\x\\ 2 . (2) 

W.l.o.g. assume that ||a;||2 = 1 and that \xi\ ^ \x%\ ^ . . . ^ \xn\. Applying Definition 12. 10^ 
we know that ||x[ t+ i..jv] H2 ^ £■ On the other hand, Yfi=i \ x i\ 2 ^ 1> therefore |x t | ^ -ft and 
thus ||^[t+i..AT]||oo ^ -ft- And now we get (j2J) by the calculation 

11 11 2 

II II ^ II II ^ \\ X [t+l..N]\\2 . /T 2 

IM|i > lF[t+i..Ar]||i ^ n n— ^ vte • 

lF[t+1..7V] II CO 

N 



b) Let t = 2A^)2- ^ix a g arn x £ X with ||x|| 2 = 1 and S C [AT] with |£| ^ £. By the 



bound on distortion, \\x\\i ^ aTZv ^ n ^ ne °^ ner hand 



|i < Vt • Ibslb < Vt 



A(X)' 



JV_ i II 11 ^ llxglli \ 1 



□ 



hence = ||x||i - ||a; 5 ||i ^ 4^ and \\xsh > ^ 4A(X) - 

Next, we note spreading properties of random subspaces (they will be needed only in the 
proof of Theorem II .2p . The following theorem is due to Kashin [23], with the optimal bound 
essentially obtained by Garnaev and Gluskin [15]. We note that such a theorem now follows 
from standard tools in asymptotic convex geometry, given the entropy bounds of Schiitt [31] 
(see, e.g. Lemma B in [2"T]). 

Theorem 2.12. If A is a uniformly random kxN sign matrix, then with probability 1 — o(l), 



A(ker(A))< J^log (j). 
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Combining Theorem 12. 121 with Lemma [2.11K b) , we get: 



Theorem 2.13. If A is a uniformly random kxN sign matrix, then with probability 1 — o(l), 
ker(A) is a (fl (j^pj) , » Nlo ^ N/k) )) spread subspace. 

Finally, we introduce a "relative" version of Definition 12.101 It is somewhat less intuitive, 
but very convenient to work with. 

Definition 2.14. A subspace X C M, N is (t, T, e)-spread (t ^T) if for every x G X, 

min llxclU ^ £ ■ min llxclU. 

SC[N] SC[N] 
S|^T |SKt 

Note that X is (i, e)-spread if and only if it is (0, t, e)-spread, if and only if it is (1/2, t, e)- 
spread. (Note that t, T are not restricted to integers in our definitions.) One obvious 
advantage of Definition 12. 141 is that it allows us to break the task of constructing well-spread 
subspaces into pieces. 

Lemma 2.15. Let X\, . . . , X r C M, N be linear subspaces, and assume that Xi is (tj-i, U, Be- 
spread, where to ^ t\ ^ • • • ^ t r . Then Di=i Xi is (to, U, n[=i £i) -spread. 

Proof. Obvious. □ 



3 An explicit weakly-spread subspace 

Now our goal can be stated as finding an explicit construction that gets as close as possible 
to the probabilistic bound of Theorem 12.131 In this section we perform a (relatively simple) 
"initialization" step; the boosting argument (which is the most essential contribution of our 
paper) is deferred to SectionlH Below, for a matrix A, we denote by \\A\\ its operator norm, 
defined as sup x#0 MJ. 

Lemma 3.1. Let A be any k x d matrix whose columns ai,...,a^ G M. h have £ 2 -norm 1, 
and, moreover, for any 1 ^ i < j ^ d, \(ai,a,j)\ ^ r. Then kei(A) is gjpjfl -spread. 

Proof. Fix x G ker(A) and let S C [d] be any subset with t = \S\ ^ Let As be the k x t 
matrix which arises by restricting A to the columns indexed by S, and let $ = A T S A$. Then $ 
is the t xt matrix whose entries are (a*, aj) for i, j G S, therefore we can write $ = /+$' where 
every entry of $' is bounded in magnitude by r. It follows that all the eigenvalues of $ lie in 
the range [1 — tr, 1 + tr]. We conclude, in particular, that ||As2/||| ^ (1 — £r)||2/||| ^ |IMIi 
for every y G M*. 

Let A§ be the restriction of A to the columns in the complement of S. Since x G ker(A), 
we have 

= Ax = A s x s + A s x s 
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so that | 

IIAsZfilh = HAszslh ^ -^=||x s || 2 . 

Since ||AgXg|| 2 ^ ||A|| ■ ^ || 2, it follows that ||xs|| 2 ^ \/2||A|| • || a:^ || 2- Since \\A\\ ^ 1, this 
implies ||2s|| 2 > |p|. □ 

We now obtain matrices with small operator norm and near-orthogonal columns from 
explicit constructions of Kerdock codes. 

Proposition 3.2. For all positive integers d, k where k is a power of 4 satisfying k ^ d ^ 
k 2 /2, there exists an explicit k x d matrix A with the following properties. 

1. Every entry of A is either ±1/V7c, and thus the columns ai,a 2 , . . . , G M. h of A all 
have £ 2 - n orm 1, 

2. For all 1 ^ i < j ^ d, \{a il aj) \ ^ 1/Vk, and 

Proof. The proof is based on a construction of mutually unbiased bases over the reals using 
Kerdock codes [251 [7]. First, let us recall that for k a power of 2, the Hadamard code of 
length k is a subspace of F 2 of size k containing the k linear functions L a : F 2 og2fc — > F 2 , 
where for a, : (computed over F 2 ). A Kerdock code is the union of a 

Hadamard code H C F 2 and a collection of its cosets {/ + H | / G JF}, where JF is a set of 
quadratic bent functions with the property that for all / 7^ g G J 7 , the function / + g is also 
bentj§ 

When is a power of 4, it is known (see [22] and also [2§1 Chap. 15, Sec. 5]) that one 
can construct an explicit set T of (| — 1) such bent functions. (A simpler construction of 
{\fk — 1) such quadratic functions appears in [7].) The cosets of these functions together 
with the Hadamard code (the trivial coset) give an explicit Kerdock code of length k that 
has k 2 /2 codewords. Interpreting binary vectors of length k as unit vectors with ±l/y/k 
entries, every coset of the Hadamard code gives an ortho normal basis of R fc . The k/2 cosets 
comprising the Kerdock code thus yield k/2 orthonormal bases Bi, B 2 , . . . , -Bfc/2 of R fc with 
the property that for every pair {v, w} of vectors in different bases, one has \(v, w} \ = 1/yk. 
(Such bases are called mutually unbiased bases.) 

For any d, k ^ d ^ k 2 /2, write d = qk + r where ^ r < k. We construct our k x d 
matrix A to consist of [B\ . . . B q ] followed by, in the case of r > 0, any r columns of B q+ \. 
The first two properties of A are immediate from the property of the bases B^. To bound 
the operator norm, note that being an orthonormal basis, ||-Bj|| = 1 for each i. A simple 
application of Cauchy-Schwartz then shows that \\A\\ ^ \J\d/k~\. □ 

Plugging in the matrices guaranteed by Proposition 13.21 into Lemma I3TT1 we can conclude 
the following. 

2 A function / : F§ — > F2 for a even is said to be bent if it is maximally far from all linear functions, or 
equivalcntly if all its Fourier coefficients have absolute value l/2 a / 2 . 
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Theorem 3.3. For every integer k that is a power of 4 and every integer d such that 

k^d^ k 2 /2, (3) 

there exists a iylj -spread subspace LCK d with codim(L) ^ k, specified as the kernel 
of an explicit k x d sign matrix. 

These subspaces will be used as "inner" subspaces in an expander-based construction 
(Theorem 14.31) to get a subspace with even better spreading properties. 

4 Boosting spreading properties via expanders 

4.1 The Tanner construction 

Definition 4.1 (Subspaces from bipartite graphs). Given a bipartite graph G = ({1, 2, . . . , N}, Vr. 
such that every vertex in Vr has degree d, and a subspace L C M. d , we define the subspace 
X = X(G,L) C R N by 

X(G, L) = {x E R N | x TgU) E L for every j e V R } . (4) 

The following claim is straightforward. 

Claim 1. Ifn= \V R \, then codim(X(G, L)) ^ codim(L)n ; that is dim(X(G, L)) ^N-(d- 
dim(L))n. 

Remark 1 (Tanner's code construction). Our construction is a continuous analog of Tanner's 
construction of error- correcting codes [37]. Tanner constructed codes by identifying the 
vertices on one side of a bipartite graph with the bits of the code and identifying the other 
side with constraints. He analyzed the performance of such codes by examining the girth 
of the bipartite graph. Sipser and Spielman |35j showed that graph expansion plays a key 
role in the quality of such codes, and gave a linear time decoding algorithm to correct a 
constant fraction of errors. In the coding world, the special case when L is the (d — 1)- 
dimensional subspace {y e M, d \ Ylt=i He = 0} corresponds to the low-density parity check 
codes of Gallager [H]. In this case, the subspace is specified as the kernel of the bipartite 
adjacency matrix of G. 

4.2 The spread-boosting theorem 

We now show how to improve spreading properties using the above construction. 

Theorem 4.2. Let G be an (N,n, D,d)- graph with expansion profile Ag{-), and let LCtf 
be a (t, e)-spread subspace. Then for every T , < T ^ N, X(G, L) is ^T , -^Ac(T ), -^7= j - 
spread. 
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Proof. Fix x G X(G, L) with ||x|| 2 = 1. Fix also S C [JV] with \S\ < T, where T = j^A G (T ). 
We then need to prove that 

e 

\\Xs 2 ^ ; min 2 - (5) 

Let 

g = {jGM:|r(j)ns|>t}, 

and 

B = {i e S : r(i) c Q} . 

Then 

t\Q\ < E(S,T(S)) ^ D\S\ < DT, 

therefore 

\Q\<^f = A G (T ) . 

On the other hand, we have \Q\ ^ and hence |r(JB)| < Aq(T ). By the definition of 

the expansion profile, this implies that \B\ < T , and therefore (see ([5])) we are only left to 
show that 

1Mb ^ • \\xsh ( 6 ) 

for our particular B. 
Note first that 

Ikslli = IMI2+ IfcsvHIl- (7) 
Next, since every vertex in S \ B has at least one neighbor in T(S) \ Q, we have 

E IkryjIll^ll^Blll. (8) 

jer(s)\g 

Since x G L), L is (t, e)-spread, and \T(J) S\ for any j G r(S') \ 

E W x m\s\\i >£ 2 - E H^dli- (9) 

jer(5)\Q ier(S)\g 

Finally, 

E II^O0\sllUElNi)\5ir 2 ^^-|k5ll2- (10) 

j&T(S)\Q j€[n] 

©-(HQD imply 

2 £ 2 2 2 

IfsL ^ ^(II x bII2 — IfsD- 

Since e ^ 1 and -D ^ 1, (JED (and hence Theorem 14. 2p follows. □ 
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4.3 Putting things together 



In this section we assemble the proofs of Theorems 11.11 and 11.21 from the already available 
blocks (which are Theorems 12. 8[ [276| 12. 13113.31 and 14. 2p . Let us first see what we can do using 
expanders from Theorem 12.81 

4.3.1 First step: Boosting with sum-product expanders 

The main difference between the explicit construction of Theorem 13.31 and the probabilistic 
result (Theorem I2.13P is the order of magnitude of t (the parameter from Definition I2.10p . 
As we will see in the next section, this difference is very principal, and our first goal is to 
somewhat close the gap with an explicit construction. 

Theorem 4.3. Fix an arbitrary constant (3q < min {0.08, §£o}; where £o is the constant 
from Theorem \2.8l Then for all sufficiently large N e N and r\ ^ jV~ 2 *' 3 there exists an 
explicit subspace X C W N with codim(X) ^ r/N which is (N^ + ^° ,7]°^) -spread. 

Proof. In everything that follows, we assume that N is sufficiently large. The desired X 
will be of the form X(G, L), where G is supplied by Theorem 12.81 and L by Theorem 13.31 
More specifically, let G be the explicit (N, n, 8, d)-right regular graph from Theorem 12.81 with 
d ~ iV 2 / 3 (and hence n ~ A 1 / 3 ). Using Theorem I2.8[ one can check that for m ^ N^ +/3 °, we 
have 

A G (m) ^ md Po ~^ . (11) 

Indeed, since n ~ A 1 / 3 and d ~ A 2 / 3 , the inequality |n ' 9 ^ md^°~^ follows (for large N) 
from p < 0.08, and the inequality m3^° ^ md^°~^ follows from (3 < §£o- 

By our assumption 77 ^ jV~ 2 ^°/ 3 ^ JV -0,1 , along with d ~ A 2 / 3 , we observe that d ^ 
o(j]d) . Hence (cf. the statement of Theorem 13.31) . we can find k < 2f, k « r]d that is a 
power of 4 and also satisfies the restrictions (j3J). Let L be an explicit [Q Cy/rja) ,Q (y/rj))' 
spread subspace guaranteed by Theorem 13.31 

The bound on codimension of X(G,L) is obvious: codim(X(G, L)) ^ kn ^ ^ ^ rjN. 

For analyzing spreading properties of X(G, L), we observe that 77 ^ jV~ 2/3 °/ 3 implies 770? > 
g? 1_/3 °, hence L is (^((i^"^ 1 ), 77°^ 1 ^)-spread. By Theorem 14.21 and (fTTI) . for every T ^ jV"2+^°, 
we know that X(G,L) is (T, f2(o^)T, ?7 ( 1 ))-spread In particular, for such T, X(G,L) is 
(T, A^MT, 77° (1) )-spread. 

Applying Lemma [2. 151 with the same spaces X\ := ■ ■ ■ := X r := X(G,L) and suitably 

large constant r « l//3 = 0(1), we conclude that X(G,L) is N^ +l3 °, r)°W j-spread, 
completing the proof. □ 

4.3.2 Second step: Handling large sets based on spectral expanders 

The sum-product expanders of Theorem [2]8] behave poorly for very large sets (i.e., as m — > A, 
the lower bound on Ac(m) becomes constant from some point). The spectral expanders of 
Theorem 12.61 behave poorly for small sets, but their expansion still improves as m — > N. 
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In this section, we finish the proofs of Theorems 11.11 and 11.21 by exploring strong sides of 
both constructions. We begin with Theorem 11.21 as it is conceptually simpler (we need only 
spectral expanders, do not rely on Theorem 14. 3[ and still use only one fixed space X(G, L)). 

Proof of Theorem 11.21 By Theorem 12.61 there exists an explicit (N, n, 4, d)-right regular 
graph G with 

]V n U°glog]v) ^ d ^ ^2 log log JV (12) 

which has A G (m) ^ min|^^, y " 2 ^ m j , Let k = [jd\; our desired (probabilistic) space is 
then X(G, ker(A)), where A is a uniformly random k x d sign matrix (due to the upper 
bound in ( fl2l) . this uses at most d 2 ^ iV 1 ^ 1 ^^ random bits). Recalling that r\ > is an 
absolute constant, by Theorem 12.131 ker (A) is an (0(<i), 0(l))-spread subspace almost surely. 

The bound on codimension is again simple: codim(X(G, ker(A))) ^ kn ^ r]N. 

For analyzing spreading properties of X, let mo = 8N/d (which is the "critical" point 
where ^ = ^Ha.) Then Theorem O says that X(G,L) is 

a. (t, tt(y/d)T, 0(1) j-spread subspace for T ^ m , and 

b. rT, 0(\/iVT), 0(1) ^-spread subspace for mo ^ T ^ N. 

And now we are once more applying Lemma f2. 151 with X\ := Xi X r :— X(G, L). 

In O(log d mo) = 0(loglogA r ) applications of (a) with T ^ m , we conclude that X(G,L) is 
(|,m , (log A^)~°( 1 ^)-spread. In 0(loglogA r ) additional applications of (b) with T ^ m , we 
conclude that X(G, L) is (|,0(iV), (log^)-°( 1 ))-spread. 

Since X(G, L) is an (0(7V), (log Af) _0(1 ))-spread subspace, the statement of Theorem II .21 



immediately follows from Lemma I2.1H a 



Proof of Theorem II. li This is our most sophisticated construction: we use a series of 
X(G, L) for different graphs G, and the "inner" spaces L will come from Theorem 14.31 In 
what follows, we assume that N is sufficiently large (obviously for N = 0(1), every non- 
trivial subspace has bounded distortion). 
To get started, let us denote 

~= v 

V (log log AT) 2 ' 

and let us first construct and analyze subspaces X(G, L) needed for our purposes individually. 
For that purpose, fix (for the time being) any value of m with 

1 ^ m ^ <ftf /3o/3 AT, (13) 

5 a sufficiently small constant and /3q is the constant from Theorem 14.31 

Applying Theorem 12.61 (with d := N/m), we get, for some d = Q(N/m), an explicit 
(N, n, 4, (i)-right regular graph G m with Ac m (m) ^ 0(c? _1 / 2 )m. Note that ( fl~3j) implies 
rj ^ dr 2 ^l z (provided the constant 5 is small enough), and thus all conditions of Theorem 14.31 
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with N := d, r\ := r\ are met. Applying that theorem, let L m C M. d be an explicit subspace 
with codim(L m ) ^ rjd that is a (d2 +/3 °, {r\j loglog A^)°( 1 ))-spread subspace. Consider the 
space X(G m ,L m ) C R*. 

Since D = 4 is a constant, we have 

codim(A(G m ,L m )) <^N ' ,S 



(log log AT) 2 ' 



And Theorem 14.21 (applied to T := m) implies (recalling Ac m (m) > d 1 / 2 m, £ = dz +f3 °, 

d = Q(N/m)) that X(G m , L m ) is a (jn, fl wi, (77/ log log X) 0(1 )j -spread subspace. 

We note that it is here that we crucially use the fact that L m has spreading properties for 
t ^> d 1 / 2 (t is the parameter from Definition 12.101) so that we more than compensate for 
the factor y/d loss in Theorem 11.21 caused by the relatively poor expansion rate of spectral 
expanders. 

We will again apply Lemma [2.151 but the spaces X* will now be distinct. In particular, 
for i 6 N define Xj = X(Gt { , L ti ), where 

for some sufficiently small constant e, < e < 1. It is easy to see that for some r = 
O(loglogJV), we have * r < 5rj 2l3o/3 N and * r > (8rf p0 ^) 2 N. 

Then for A = 01=0 ^ we have codim(X) < r {log ^ gN)2 < j^^- In particular, 
codim(A) ^ rjN for sufficiently large N. 

By the above argument based on Theorem 14.21 and the choice of the t^s, it is easily 
seen that Xj is a (U, U+i, (?y/ (log log A))°( 1 ^ ) )-spread subspace. By Lemma [2.151 A is a 
(e, U , (V (log log A))° (log lo s N *> )-spread subspace, or equivalently a (t r , (77/ (log log N) lo s ^ ) 
spread subspace. Since we also have t r ^ (?y/ (log log N) N, the required bound on A(A) 
follows from Lemma 12. 11 ( la)] 



5 Discussion 



We have presented explicit subspaces A C WL N of dimension (1 — rj)N with distortion 
(7/- 1 loglogJV)°( log]ogJV 5 and, using X°W random bits, distortion v ~o(io g i og N) _ It ig natu _ 

ral to wonder whether better explicit constructions of expanders can give rise to better 
bounds. We make some remarks about this possibility. 

1. The GUV and CRVW expander families. The next two theorems essentially 
follow from [T7| and [8], respectively (after an appropriate application of Lemma [2. 3p . 

Theorem 5.1 (|17J). For each fixed < c, e ^ 1, and for all integers N, K with K ^ 
N , there is an explicit construction of an (N,n, D,d) -right regular graph G with D < 
((log N)/e) 2+2 ' c and d ^ N/{DK l+c ) and such that A G (m) > (1 - e)D ■ min{A,m}. 
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Theorem 5.2 Q8j). For every fixed < e < 1 and all sufficiently large values N and 
d there exist n ^ N, D ^ 2°^ £ Mogiogo!) 3 ) an( ^ an ex pH c n (N,n, D,d) -right regular 
bipartite graph G with Ac-(m) ^ (1 — e)D ■ min {Q(N/d), m}. 

The main problem for us in both these constructions is that D must grow with N and 
d, respectively. By plugging in the explicit subspaces of Theorem 13.31 into Theorem 
14.21 with the GUV-expanders from Theorem 15. 1\ one can achieve distortions A(X) « 
exptyiogiVlog logiV) for X C R N with dim(X) ^ N/2. Using the GUV-expanders (in 
place of the sum-product expanders) together with spectral expanders in a construction 
similar to the proof of Theorem 11.11 would yield a distortion bound of (log N)°( loglogN \ 

2. Very good expansion for large sets. If it were possible to construct an (N, n, D, d)- 
right regular bipartite graph H with D = 0(1) and such that for every S C Vl with 
\S\ ^ iV 1-/3 , we had |r(5)| = Q(n), then we would be able to achieve 0(1) distortion 
using only 0(d 2 + N s ) random bits for any S > (in fact, we could use only 0(d + N s ) 
random bits with [2"T]). 



The idea would be to follow the proof of Theorem 11.21 but only for 0(1) steps to show 
the subspace is (iV 1_/3 , f2(l))-spread. Then we would intersect this with a subspace 
X(H, L), where L C M. d , with the latter subspace generated as the kernel of a random 
sign matrix (requiring d 2 bits). Unfortunately, [311 Th. 1.5] shows that in order to 
achieve the required expansion property, one has to take D ^ Q(/3logN). 
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